Training label image correction method, trained model creation method, and image analysis device

ABSTRACT

A training label image correction method includes performing a segmentation process on an input image ( 11 ) of training data ( 10 ) by a trained model ( 1 ) using the training data to create a determination label image ( 14 ), comparing labels of corresponding portions in the determination label image ( 14 ) and a training label image ( 12 ) with each other, and correcting label areas ( 13 ) included in the training label image based on label comparison results.

TECHNICAL FIELD

The present invention relates to a training label image correctionmethod, a trained model creation method, and an image analysis device.

BACKGROUND ART

Conventionally, training label image correction in machine learning isknown. Such training label image correction is disclosed in JapanesePatent Laid-Open No. 2014-022837, for example.

Japanese Patent Laid-Open No. 2014-022837 discloses that a classifier(trained model) to perform image classification that assigns a labelindicating whether a detection target appears (a positive example) ordoes not appear (a negative example) in a certain video section in videodata such as a TV program is constructed by machine learning. Trainingdata including video and a label of whether it is a positive example ora negative example is used to construct the classifier. The label of thetraining data may contain errors and/or omissions. Therefore, JapanesePatent Laid-Open No. 2014-022837 discloses that the label of the initialtraining data or the label of the training data in which the label hasbeen set is corrected based on an user input or the result of detectionof the training data by another classifier.

PRIOR ART Patent Document

-   -   Patent Document 1: Japanese Patent Laid-Open No. 2014-022837

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

The classifier disclosed in Japanese Patent Laid-Open No. 2014-022837simply classifies the input video and labels the input video in units ofvideo, but as a technique different from such image segmentation, thereis image segmentation. The image segmentation is a technique fordividing an image into a plurality of areas instead of the entire image,and divides an input image into a plurality of label areas by assigninga label indicating a detection target to an area in which the detectiontarget appears. Note that the “image segmentation” in this specificationis called semantic segmentation because a meaning is given to an area bya label, unlike simple area division (boundary detection) that does notassign a label such as edge detection.

In machine learning (supervised learning) for constructing a trainedmodel used for image segmentation, an input image and a training labelimage in which the input image has been divided into areas for eachlabel are used. The training label image in this case is created byextracting the label areas from an original image by a thresholdprocess, for example. There is a variation in the appearance (such asthe signal strength) of the detection target in each input image, andthus creation of the training label image is shared by a large number oflabel creators, and each label creator manually adjusts a threshold tocreate the training label image.

Consequently, the creation of the training label image includes anindividual difference in threshold setting, for example, and a certainlabel area may be extracted to a large or small extent. Furthermore,there is a case in which the boundaries of the label areas are ambiguousin the first place depending on an image, and in that case, theboundaries of the label areas tend to vary. Therefore, the boundaries inthe training label image are corrected.

However, unlike the simple label correction for each image as disclosedin Japanese Patent Laid-Open No. 2014-022837, in order to correct thevariations in the boundaries of the label areas in the training labelimage, it is desirable to correct the label areas with consistentcriteria for all training label images. Therefore, for example, it takesa lot of effort such as making corrections only by a specific operator.It is also possible to create a dedicated trained model for correcting atraining label image, but this requires learning a highly accuratetraining label image created with consistent criteria, and thussimilarly, it takes a lot of effort. Therefore, it is desired thatcorrection of the training label image used for machine learning of theimage segmentation process can be easily performed while ensuring theaccuracy.

The present invention is intended to solve at least one of the aboveproblems. The present invention aims to provide a training label imagecorrection method, a trained model creation method, and an imageanalysis device capable of easily performing correction of a traininglabel image used for machine learning of an image segmentation processwhile ensuring the accuracy.

Means for Solving the Problems

In order to attain the aforementioned object, as a result of earneststudies, the inventors have found that even in the case of traininglabel images including variations in the boundaries of label areas, atrained model that has undergone sufficient machine learning using thosetraining label images produces intermediate segmentation resultsobtained by statistically averaging variations in the boundaries of manytraining label images, and thus the segmentation results by the trainedmodel can be consistent criteria for the variations in the boundaries inthe training label image. Based on this finding, the training labelimage correction method according to a first aspect of the presentinvention is a training label image correction method in machinelearning for performing a segmentation process on an image, and includesperforming the segmentation process on an input image of training dataincluding the input image and a training label image by a trained modelusing the training data to create a determination label image dividedinto a plurality of label areas, comparing labels of correspondingportions in the created determination label image and the training labelimage with each other, and correcting the label areas included in thetraining label image based on label comparison results.

In the training label image correction method according to the firstaspect of the present invention, with the configuration described above,the training label image can be corrected based on a comparison betweenthe determination label image created by the trained model trained usingthe training data including the training label image containingvariations in the boundaries of the label areas and the training labelimage. That is, under ordinary circumstances, machine learning isperformed on the premise that the label areas of the training labelimage are correct, and the trained model is created. However, in thesegmentation results (determination label image) using the trained modelthat has undergone sufficient training, intermediate results ofvariations in the boundaries of many training label images are obtainedwith respect to the boundary portions of the label areas, and thus thevalidity of the boundaries in the determination label image can beconsidered to be higher than that in the individual training label imageincluding the variations. Therefore, the label areas of the traininglabel image are corrected based on the comparison results between thedetermination label image and the training label image with reference tothe determination label image such that the variations in the boundariescan be reduced based on consistent criteria (the boundaries of the labelareas of the determination label image) by automatic correction using acomputer instead of a label correction operator. Accordingly, correctionof the training label image used for machine learning of the imagesegmentation process can be simplified while ensuring the accuracy.

In the aforementioned training label image correction method accordingto the first aspect, the correcting the label areas may includecorrecting ranges of the label areas in the training label image so asto be closer to the label areas of the determination label image basedon the label comparison results. Accordingly, the ranges of the labelareas in the training label image are simply matched with the labelareas of the determination label image or are simply set to intermediateranges between the label areas of the training label image and the labelareas of the determination label image, for example, such that acorrection can be easily and effectively made to reduce the variationsin the label areas of the individual training label image.

In this case, the correcting the label areas may include correcting thelabel areas by at least one of expansion or contraction of the labelareas in the training label image. Accordingly, the label areas of thetraining label image can be corrected by a simple process of expandingor contracting the ranges of the label areas in the training label imagebased on the label comparison results. The expansion of the label areasindicates increasing the areas of the label areas, and the contractionof the label areas indicates reducing the areas of the label areas.

In the aforementioned training label image correction method accordingto the first aspect, the comparing the labels may include acquiring amatching portion and a non-matching portion with the training labelimage in the determination label image for a label of interest bycomparing the determination label image with the training label image,and the correcting the label areas may include correcting a range of alabel area with the label of interest based on the matching portion andthe non-matching portion. Accordingly, it is possible to understand,from the matching portions and the non-matching portions between thedetermination label image and the training label image, how the labelarea of the training label image deviates from the determination labelimage that serves as a reference for variations in the boundaries ofmany training label images. Therefore, the label area can be easilycorrected based on the matching portion and the non-matching portionsuch that a variation in the boundary is reduced.

In this case, the comparing the labels may include acquiring anon-detection evaluation value for evaluating undetected labels in thedetermination label image and a false-detection evaluation value forevaluating falsely detected labels in the determination label imagebased on the matching portion and the non-matching portion, and thecorrecting the label areas may include correcting the label areas of thetraining label image based on a comparison between the non-detectionevaluation value and the false-detection evaluation value. Non-detectionindicates that a portion with a label of interest in the training labelimage has not been detected as the corresponding label area in thedetermination label image, and when there are many undetected areas, itis estimated that the label areas of the training label image are largerthan the label areas of the determination label image. False detectionindicates that a different label has been assigned to a portion in thetraining label image corresponding to a portion detected as a label areaof interest in the determination label image, and when there are manyfalsely detected areas, it is estimated that the label areas of thetraining label image are smaller than the label areas of thedetermination label image. Therefore, the non-detection evaluation valueand the false-detection evaluation value are compared such that it ispossible to understand how the label areas of the training label imageshould be corrected (whether it should be increased or decreased), andthus the label areas of the individual training label image can be moreappropriately corrected.

In the aforementioned configuration including acquiring thenon-detection evaluation value and the false-detection evaluation value,the correcting the label areas may include expanding the label areas ofthe training label image when it is determined by the comparison betweenthe non-detection evaluation value and the false-detection evaluationvalue that there are more falsely detected labels than undetected labelsin the determination label image. Accordingly, when there are manyfalsely detected areas and it is estimated that the label areas of thetraining label image are smaller than the label areas of the referencedetermination label image, the label areas of the training label imageare expanded such that a correction can be easily made to reduce thevariations in the boundaries of the label areas.

In the aforementioned configuration including acquiring thenon-detection evaluation value and the false-detection evaluation value,the correcting the label areas may include contracting the label areasof the training label image when it is determined by the comparisonbetween the non-detection evaluation value and the false-detectionevaluation value that there are more undetected labels than falselydetected labels in the determination label image. Accordingly, whenthere are many undetected areas and it is estimated that the label areasof the training label image are larger than the label areas of thereference determination label image, the label areas of the traininglabel image are contracted such that a correction can be easily made toreduce the variations in the boundaries of the label areas.

The aforementioned configuration including acquiring the non-detectionevaluation value and the false-detection evaluation value may furtherinclude excluding the training data including the training label imagedetermined to have at least one of more than a predetermined thresholdnumber of undetected labels or more than a predetermined thresholdnumber of falsely detected labels from training data set. Accordingly,when at least one of the number of undetected labels or the number offalsely detected labels is excessively large in the comparison with thereference determination label image, the training label image isconsidered to be inappropriate as the training data. Therefore, thetraining label image in which at least one of the number of undetectedlabels or the number of falsely detected labels is large is excludedfrom the training data set such that the training data corresponding toa so-called statistical outlier that is a factor that lowers theaccuracy of machine learning can be excluded. Thus, the quality of thetraining data set can be improved, and highly accurate machine learningcan be performed.

The aforementioned training label image correction method according tothe first aspect may further include setting priorities between labelsof the training label image, and the correcting the label areas mayinclude adjusting amounts of correction of the label areas according tothe priorities. The segmentation process is to distinguish and label anarea in an image in which a detection target appears, and thus differentpriorities can be set for the labels according to the purpose of theprocess in order to distinguish between the detection target (priority:high) and a non-detection target (priority: low), for example.Therefore, with the configuration described above, the amount ofcorrection of a label with a higher priority can be intentionally biasedsuch that detection omissions can be significantly reduced or preventedas much as possible, for example. Consequently, the training label imagecan be appropriately corrected according to the purpose of thesegmentation process.

In this case, the correcting the label areas may include correcting thelabel areas of the training label image while expansion of the labelareas having a lower priority to the label areas having a higherpriority is prohibited. Accordingly, even when the label areas having alower priority are corrected by expansion, the label areas having ahigher priority can be preserved. Therefore, it is possible to obtainthe training label image (training data) capable of being learned, inwhich detection omissions of the label areas with a higher priority aresignificantly reduced or prevented while the variations in theboundaries are significantly reduced or prevented.

In the aforementioned training label image correction method accordingto the first aspect, the correcting the label areas may include dividingthe training label image into a plurality of partial images, andcorrecting the label areas for at least one partial image of the dividedtraining label image. Accordingly, only a specific portion of thetraining label image can be corrected, or each small portion can becorrected individually.

In the aforementioned training label image correction method accordingto the first aspect, the comparing the labels may include comparing thelabels of the corresponding portions of the determination label imageand the training label image with each other for every image pixel orevery plurality of adjacent image pixels. Accordingly, the determinationlabel image and the training label image can be compared with each otherfor each small area of a unit of one pixel or a unit of multiple pixels,and thus the variations in the label areas of the training label imagecan be evaluated more accurately.

In the aforementioned training label image correction method accordingto the first aspect may further include creating the determination labelimage by the trained model using the training data including thecorrected training label image, and the correcting the label areas mayinclude correcting the label areas included in the corrected traininglabel image again based on comparison results between the createddetermination label image and the corrected training label image.Accordingly, machine learning is performed using the corrected traininglabel image, and then the determination label image is created againsuch that the label areas of the training label image can be repeatedlycorrected. The label areas are repeatedly corrected such that thevariations in the label areas of the training label image can be furtherreduced.

A trained model creation method according to a second aspect of thepresent invention is a trained model creation method by machine learningfor performing a segmentation process on an image, and includesacquiring a pre-trained model by the machine learning using trainingdata including an input image and a training label image, performing thesegmentation process on the input image of the training data by theacquired pre-trained model to create a determination label image dividedinto a plurality of label areas, comparing labels of correspondingportions in the created determination label image and the training labelimage with each other, correcting the label areas included in thetraining label image based on label comparison results, and creating atrained model by performing the machine learning using the training dataincluding the corrected training label image. The creating the trainedmodel can include both (1) creating the trained model by performing themachine learning (additional training) on the pre-trained model usingthe training data including the corrected training label image and (2)creating the trained model by performing the machine learning on anuntrained training model using the training data including the correctedtraining label image.

In the trained model creation method according to the second aspect ofthe present invention, similarly to the first aspect, the label areas ofthe training label image are corrected based on the comparison resultsbetween the determination label image and the training label image withreference to the determination label image such that the variations inthe boundaries can be reduced based on consistent criteria (theboundaries of the label areas of the determination label image) even byautomatic correction using a computer. Accordingly, correction of thetraining label image used for machine learning of the image segmentationprocess can be simplified while ensuring the accuracy. Furthermore, themachine learning is performed on the pre-trained model using thetraining data including the corrected training label image such that thetrained model with significantly reduced determination variations in theboundary portions of the label areas, which is capable of a high-qualitysegmentation process, can be obtained.

An image analysis device according to a third aspect of the presentinvention includes an image input configured to receive an input of ananalysis image, an analysis processor configured to perform asegmentation process on the analysis image using a trained model bymachine learning to create a label image divided into a plurality oflabel areas, a storage configured to store the trained model, adetermination image creating unit configured to perform the segmentationprocess on an input image of training data including the input image anda training label image with the trained model stored in the storage as apre-trained model to create a determination label image, a comparatorconfigured to compare labels of corresponding portions in thedetermination label image created by the determination image creatingunit and the training label image with each other, and a label correctorconfigured to correct the label areas included in the training labelimage based on label comparison results by the comparator.

In the image analysis device according to the third aspect of thepresent invention, similarly to the first aspect, the label areas of thetraining label image are corrected based on the comparison resultsbetween the determination label image and the training label image withreference to the determination label image created by the pre-trainedmodel such that the variations in the boundaries can be reduced based onconsistent criteria (the boundaries of the label areas of thedetermination label image) even by automatic correction using acomputer. Accordingly, correction of the training label image used formachine learning of the image segmentation process can be simplifiedwhile ensuring the accuracy.

Effect of the Invention

According to the present invention, as described above, it is possibleto easily perform the correction of the training label image used formachine learning of the image segmentation process while ensuring theaccuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for illustrating a training label image correctionmethod according to a first embodiment.

FIG. 2 is a diagram showing the overview of machine learning and asegmentation process.

FIG. 3 is a diagram showing examples of an X-ray image of a bone and alabel image of two classes.

FIG. 4 is a diagram showing examples of a cell image and a label imageof three classes.

FIG. 5 is a diagram for illustrating a training label image.

FIG. 6 is a graph for illustrating a change in the loss function with anincrease in the number of times of learning.

FIG. 7 is a diagram showing examples of an input image, a determinationimage, and the training label image.

FIG. 8 is a diagram for illustrating a variation in the boundary of alabel area of the training label image.

FIG. 9 is a diagram for illustrating correction of label areas of thetraining label image.

FIG. 10 is a flowchart for illustrating a method for correcting thelabel areas of the training label image.

FIG. 11 is a diagram for illustrating a method for comparing thetraining label image with the determination image.

FIG. 12 is a schematic view showing an example of contracting the labelarea of the training label image.

FIG. 13 is a schematic view showing an example of expanding the labelarea of the training label image.

FIG. 14 is a flowchart for illustrating a trained model creation method.

FIG. 15 is a block diagram for illustrating a training data processor.

FIG. 16 is a block diagram showing a first example of an image analysisdevice.

FIG. 17 is a flowchart for illustrating an image analysis process of theimage analysis device.

FIG. 18 is a block diagram showing a second example of an image analysisdevice.

FIG. 19 is a diagram for illustrating the label priorities in a secondembodiment.

FIG. 20 is a diagram for illustrating adjustment of the amounts ofcorrection of label areas according to the label priorities.

FIG. 21 is a diagram showing a modified example in which label areacorrection is performed for each partial image.

FIG. 22 is a schematic view showing a modified example in which traininglabel image correction and trained data creation are performed on theserver side.

FIG. 23 is a schematic view showing a modified example in which traininglabel image correction is performed on the image analysis device sideand trained data creation is performed on the server side.

MODES FOR CARRYING OUT THE INVENTION

Embodiments embodying the present invention are hereinafter described onthe basis of the drawings.

A training label image correction method, a trained model creationmethod, and an image analysis device according to a first embodiment arenow described with reference to FIGS. 1 to 13 .

The training label image correction method according to the firstembodiment shown in FIG. 1 is a method for correcting a training labelimage in machine learning for performing an image segmentation process.Hereinafter, performing the segmentation process may be paraphrased as“performing area division”.

As shown in FIG. 2 , the image segmentation process is performed by atrained model 2 created by machine learning. The trained model 2performs the segmentation process on an input image (an analysis image15 or an input image 11) and outputs a label image (a label image 16 ora determination label image 14) divided into a plurality of label areas13 (see FIGS. 3 and 4 ). As a machine learning method, any method suchas a fully convolutional network (FCN), a neural network, a supportvector machine (SVM), or boosting can be used. For the trained model 2in the first embodiment, from the viewpoint of the identificationperformance of the label areas, it is preferable to use a convolutionalnetwork frequently used for semantic segmentation, and it is morepreferable to use a fully convolutional network. Such a trained model 2includes an input layer into which an image is input, a convolutionlayer, and an output layer.

In order to create a trained model that performs the image segmentationprocess, machine learning is performed using a training data set 1 (seeFIG. 1 ) that includes a large number of training data 10.

The training data 10 used for machine learning includes at least theinput image 11 and a training label image 12. The input image 11 is anoriginal image before performing the segmentation process. The traininglabel image 12 is created as a correct image to be generated as a resultof the segmentation process on the input image 11. That is, the traininglabel image 12 is a label image obtained by dividing the input image 11into a plurality of label areas 13. Each of the label areas 13 is anarea (a portion of an image) including a group of pixels with a commonlabel in an image.

The label is information indicating a meaning indicated by an imageportion including the label area 13. Segmentation is performed byassigning a label to each pixel in an image. The label may be assignedin units of a group (pixel group) of a plurality of pixels. The type oflabel is called a class.

The number of classes (the types of labels) is not particularly limitedas long as there are a plurality of (two or more) classes. In thesimplest example, the classes include two classes, a “detection target”label and a “non-detection target” label. For example, in FIG. 3 , anexample is shown in which using an X-ray image of the human pelvis (abase of the femur) as the input image 11, area division into a labelarea 13 a of a “bone” as a detection target and a label area 13 b of a“portion other than the bone” has been performed.

FIG. 4 shows an example in which using an image (cell image) of apluripotent stem cell, such as an iPS cell or an ES cell, as the inputimage 11, area division into three classes is performed. In FIG. 4 ,area division into three classes including a label area 13 c of an“undifferentiated cell”, which is a cell that maintains pluripotency, alabel area 13 d of an “undifferentiated deviant cell”, which is a cellthat has deviated from the undifferentiated state (a cell that hasstarted differentiation or is likely to differentiate), and a label area13 e of a “background” other than those has been performed.

When the trained model is created, the input image 11 is input to atraining model, the training label image 12 is output, and a conversionprocess (segmentation process) from the input image 11 to the traininglabel image 12, which is a correct answer, is learned by the trainingmodel. The trained model 2 capable of the segmentation process isgenerated by performing machine learning. Consequently, the analysisimage 15 to be analyzed is input into the trained model 2 such that thesegmentation process for the analysis image 15 is performed using thetrained model 2 by machine learning, and the label image divided intothe plurality of label areas 13 is output.

<Training Label Image>

FIG. 5 shows an example of creating the training label image 12 in whicharea division into three classes has been performed with respect to theinput image 11 of the pluripotent stem cell shown in FIG. 4 . In theexample of FIG. 5 , for a pluripotent stem cell, a cell membranestaining image 91 in which a cell area has been stained with a stainingagent and a nuclear staining image 92 in which a nuclear staining areaof an undifferentiated cell has been stained with an undifferentiatedmarker are acquired, and after the cell membrane staining image 91 andthe nuclear staining image 92 are binarized by a threshold process, adifference between the two images is acquired such that the traininglabel image 12 is created.

In such a training label image 12, the training label image 12 iscreated based on a staining image of a cell, and thus there is aninevitable variation in the degree of staining. Therefore, it isnecessary to manually adjust a threshold according to the degree ofstaining, and depending on setting of the threshold, a boundary portionof the cell becomes relatively large or small with respect to anothertraining label image 12 and varies. In addition, whereas in the cellmembrane staining image 91, a cytoskeleton as well as a nucleus isstained, in the nuclear staining image 92, only a nucleus is stained,and thus a stained area may differ slightly between an undifferentiateddeviant cell and an undifferentiated cell. Such a difference may causevariations in the boundaries (sizes) of the label areas 13 in thetraining label image 12. In the X-ray image of the bone shown in FIG. 3, there is no difference in a stained area due to a difference in astaining method, but variations in the boundary portions due to thethreshold process occur similarly to the example of the cell image.

When the boundaries of the label areas 13 in an individual traininglabel image 12 vary, the criteria of which portions in an image aredefined as boundaries varies. Therefore, in the trained model 2 that hasbeen trained using the training data set 1 including the training labelimage 12 in which the boundaries of the label areas 13 have varied, theboundaries of the label areas 13 may be blurred or may have an unnaturalshape in the generated label image 16 (see FIG. 2 ).

Therefore, in the first embodiment, as shown in FIG. 1 , the trainedmodel 2 (hereinafter referred to as a pre-trained model 2 a) trainedusing the training data 10 including the training label image 12 inwhich the boundaries of the label areas 13 have varied is used tocorrect the label areas 13 of the training label image 12.

(Method for Correcting Training Label Image)

Specifically, the method for correcting the training label image in thefirst embodiment includes the following steps as shown in FIG. 1 :

-   -   (1) a step of performing the segmentation process on the input        image 11 of the training data 10 including the input image 11        and the training label image 12 by the trained model 2 using the        training data 10 to create the determination label image 14        divided into a plurality of label areas 13,    -   (2) a step of comparing the labels of the corresponding portions        in the created determination label image 14 and the training        label image 12 with each other, and    -   (3) a step of correcting the label areas 13 included in the        training label image 12 based on the label comparison results.

In the first embodiment, the pre-trained model 2 a that generates thedetermination label image 14 is a trained model trained using thetraining data 10 including the training label image 12 in which theboundaries of the label areas 13 vary.

<Trained Model and Pre-Trained Model>

As the trained model 2 (pre-trained model 2 a) that generates thedetermination label image 14, a trained model that has been trainedusing the training data set 1 including the training data 10 having asufficient score and in which the training has appropriately convergedis used. That is, in the pre-trained model 2 a, machine learning itselfis appropriately performed, and the segmentation process can beperformed with sufficient accuracy except for the boundaries of thelabel areas 13. In the field of machine learning, the accuracy of thepre-trained model 2 a can be evaluated by obtaining an error (lossfunction) between the segmentation result (here, the determination labelimage 14) for each input image 11 and the training label image 12 of theinput image 11. As shown in FIG. 6 , when training is appropriatelyperformed with the training data set 1 having a sufficient score, in thetrained model, a value of the loss function decreases with the number oftimes of training, and converges at a value corresponding to a limitcaused by various inevitable factors including the variations in theboundaries of the label areas 13. In a graph of FIG. 6 , the verticalaxis represents the loss function logarithmically, and the horizontalaxis represents the number of training iterations.

The training label image 12 is a training label image created accordingto consistent criteria to the extent that training of the pre-trainedmodel 2 a can appropriately converge. That is, a large number oftraining label images 12 included in the training data set 1 havevariations in the boundaries of the label areas 13 due to a factor suchas the above threshold process, but statistical bias is small enough toallow training to appropriately converge. As can be seen from the imagesof the pelvis shown in FIG. 3 and the cell images shown in FIG. 4 , aratio of the boundary portions to the entire label areas 13 is verysmall, and the variations in the boundaries do not interfere withtraining convergence.

<Details of Method for Correcting Training Label Image>

In the step of creating the determination label image 14, the inputimage 11 is input to the pre-trained model 2 a, and the segmentationprocess is performed such that the determination label image 14 iscreated as an output. As shown in FIG. 7 , the determination label image14 is an image obtained by dividing the input image 11 into a pluralityof label areas 13, similarly to the training label image 12.

In the step of comparing the labels of the determination label image 14and the training label image 12, the labels of the correspondingportions in the determination label image 14 and the training labelimage 12 are compared with each other. The corresponding portions referto portions that can be regarded as images of the same portions in thedetermination label image 14 and the training label image 12, andindicate the same coordinates as long as the imaging field of view ofeach image is the same.

The label comparison between the corresponding portions of thedetermination label image 14 and the training label image 12 can beperformed for every image pixel or for every plurality of adjacent imagepixels. The plurality of adjacent pixels include a certain pixel andsurrounding pixels, and may be a square area of 4 (2×2) pixels or 9(3×3) pixels, for example. In the first embodiment, preferably, thelabels of the corresponding portions of the determination label image 14and the training label image 12 are compared with each other for eachpixel.

In the first embodiment, in the step of comparing the labels, a matchingportion and a non-matching portion of the determination label image 14with the training label image 12 are acquired for a label of interest bycomparing the determination label image 14 with the training label image12.

In the step of correcting the label areas 13 included in the traininglabel image 12, a range of the label area 13 with the label of interestis corrected based on the matching portion and the non-matching portionas the comparison results. The label areas 13 included in the traininglabel image 12 are corrected by replacing the labels assigned to thelabel areas 13 to be corrected with other labels, for example. As shownin FIG. 1 , the label areas 13 are corrected such that a correctedtraining label image 12 a is created. The created corrected traininglabel image 12 a is replaced with the original (uncorrected) traininglabel image 12 in the training data 10 (that is, the training labelimage 12 is updated).

When the training data set 1 allows the training of the pre-trainedmodel 2 a to appropriately converge, it is expected that the traininglabel image 12 in which the boundaries of the label areas 13 vary in adirection in which the label areas 13 become relatively large and thetraining label image 12 in which the boundaries of the label areas 13vary in a direction in which the label areas 13 become relatively smallare distributed substantially evenly (exhibit substantially normaldistribution), as shown in FIG. 8 . The horizontal axis of FIG. 8represents the magnitude of the variation, and the vertical axisrepresents the number (frequency) of training label images having thecorresponding variation. As a result of learning the training labelimage 12 in which the variations are distributed as shown in FIG. 8 ,the boundaries of the label areas 13 of the determination label image 14output by the pre-trained model 2 a are expected to be consistentlyintermediate results in the distribution of FIG. 8 . Therefore, it canbe considered that the boundary portions of the label areas 13 of theindividual training label image 12 vary by differences from thedetermination label image 14 based on the label areas 13 of thedetermination label image 14 generated by the pre-trained model 2 a.

Therefore, in the first embodiment, in the step of correcting the labelareas 13, the ranges of the label areas 13 in the training label image12 are brought closer to the label areas 13 of the determination labelimage 14 as shown in FIG. 9 based on the comparison results of thelabels. Specifically, the correction of the label areas 13 is performedby at least one of expansion or contraction of the label areas 13 in thetraining label image 12. That is, for the training label image 12 inwhich the label area 13 is set relatively large (see a boundary 18 b)with respect to a boundary 18 a of the label area 13 of thedetermination label image 14, a correction is made to contract the labelarea 13. For the training label image 12 in which the label area 13 isset relatively small (see a boundary 18 c) with respect to the boundary18 a of the label area 13 of the determination label image 14, acorrection is made to contract the label area 13. Consequently, it ispossible to reduce the variations in the boundary portions of the labelareas 13 of the individual training label image 12 without the entiretraining data set 1 being biased.

The label areas 13 can be corrected by a morphology process, forexample. The morphology process determines pixel values of correspondingpixels in the output image (here, the corrected training label image 12a) based on a comparison between pixels of the input image (here, thetraining label image 12) and pixels adjacent thereto. When an expansionprocess is performed, the same labels as the label areas 13 are assignedto pixels at the boundaries of the label areas 13 detected by thecomparison with the adjacent pixels. When a contract process isperformed, the label is removed from the pixels at the boundary of thelabel area 13 detected by the comparison with the adjacent pixels, andthe same label as the label outside the boundary is assigned. The amountof correction by the morphology process (how many pixels to expand orcontract the boundary) is not particularly limited, and a correction canbe made for one pixel or a plurality of pixels.

<Specific Example of Correcting Training Label Image>

A specific example of correcting the training label image is now shown.As shown in FIG. 10 , in step S1, the training data 10 including thetraining label image 12 to be corrected is selected from the trainingdata set 1.

Step S2 is the above step of creating the determination label image 14.In step S2, the determination label image 14 is created from the inputimage 11 by the pre-trained model 2 a.

Step S3 is the above step of comparing the labels of the determinationlabel image 14 and the training label image 12. As a method forcomparing the training label image 12 with the determination label image14, a confusion matrix 21 of two-class (positive and negative)classification shown in FIG. 11 is illustrated. The vertical directionof the matrix shows the result of the segmentation in the training labelimage 12, and the horizontal direction of the matrix shows the result ofthe segmentation in the determination label image 14. TP represents thetotal number of matching pixels between the determination label image 14and the training label image 12 to which a “positive” label has beenassigned. TN represents the total number of matching pixels between thedetermination label image 14 and the training label image 12 to which a“negative” label has been assigned. Therefore, TP and TN each representthe number of pixels in the matching portion. On the other hand, FPrepresents the total number of pixels to which a “positive” label hasbeen assigned in the determination label image 14 and a “negative” labelhas been assigned in the training label image 12. FN represents thetotal number of pixels to which a “negative” label has been assigned inthe determination label image 14 and a “positive” label has beenassigned in the training label image 12. Therefore, FP and FN eachrepresent the number of pixels in the non-matching portion.

In the first embodiment, an evaluation value for evaluating the degreeof deviation between the determination label image 14 and the traininglabel image 12 is calculated. Specifically, in the step of comparing thelabels, a non-detection evaluation value 22 for evaluating undetectedlabels and a false-detection evaluation value 23 for evaluating falselydetected labels in the determination label image 14 are acquired basedon the matching portion and the non-matching portion. In the firstembodiment, the label areas 13 of the training label image 12 arecorrected (assuming that it is incorrect), but the non-detectionevaluation value 22 and the false-detection evaluation value 23 areevaluation values for evaluating the undetected and falsely detectedlabels in the determination label image 14 assuming that each label area13 of the training label image 12 is correct. Assuming that each labelarea 13 of the training label image 12 is correct, the non-matchingportion with the training label image 12 in the determination labelimage 14 is a portion that is undetected or falsely detected. Therefore,the non-detection evaluation value 22 and the false-detection evaluationvalue 23 are values for numerically evaluating matching and non-matchingwith the training label image 12 in the determination label image 14.

The non-detection evaluation value 22 and the false-detection evaluationvalue 23 are not particularly limited as long as the same are numericalvalues indicating many (or few) undetected labels and many (or few)falsely detected labels, respectively.

As an example, the non-detection evaluation value 22 is a detection rate(recall or true positive rate; TPR) expressed by the following formula(1). The detection rate is also called sensitivity.K=TP/(TP+FN)  (1)

The detection rate refers to a “ratio of items (pixels) that can beclassified as positive correctly to items (pixels) that should beclassified as positive”, and indicates few undetected labels. That is,when the detection rate is used as the non-detection evaluation value22, the larger value indicates fewer undetected labels.

The false-detection evaluation value 23 is a precision expressed by thefollowing formula (2).G=TP/(TP+FP)  (2)

The precision refers to a “ratio of items that are actually positiveamong the items classified as positive”, and indicates a small number offalsely detected labels. That is, when the precision is used as thefalse-detection evaluation value 23, the larger value indicates fewerfalsely detected labels.

It is possible to determine whether the label areas 13 of the traininglabel image 12 are set relatively large or the label areas 13 are setrelatively small by a comparison of the non-detection evaluation value22 and the false-detection evaluation value 23. Therefore, in the stepof correcting the label areas 13, the label areas 13 of the traininglabel image 12 are corrected based on the comparison of thenon-detection evaluation value 22 and the false-detection evaluationvalue 23.

The relationship between the comparison results of the non-detectionevaluation value 22 and the false-detection evaluation value 23 and thesize of each label area of the determination label image 14 and thetraining label image 12 is described using simplified virtual examplesshown in FIGS. 12 and 13 . In FIGS. 12 and 13 , a colored portion showsthe label area 13 of each image, and a hatched portion shows a labelarea 13 f of the determination label image 14 superimposed on thetraining label image 12.

As shown in FIG. 12 , in a case in which the determination label image14 and the training label image 12 are compared, and there are manyundetected labels (the detection rate K is small), a label area 13 gwith a positive label in the training label image 12 is larger than thelabel area 13 f with a positive label in the determination label image14, and the boundary of the label area 13 g is outside the label area 13f (the outside of the label area 13 f is undetected).

Therefore, when it is determined by the comparison of the non-detectionevaluation value 22 and the false-detection evaluation value 23 that thenumber of undetected labels is larger than the number of falselydetected labels in the determination label image 14, in step S4 ofcorrecting the label areas 13, the label areas 13 of the training labelimage 12 are contracted.

That is, when detection rate K<precision G≈1, it can be determined thatthere are few falsely detected labels and many undetected labels.Therefore, the ranges of the label areas 13 of the training label image12 are contracted such that it can be brought closer to the label areas13 of the determination label image 14. In the virtual example of FIG.12 , K=0.56 and G=1, and thus the condition (detection rate K<precisionG≈1) is satisfied.

On the other hand, as shown in FIG. 13 , in a case in which thedetermination label image 14 and the training label image 12 arecompared, and there are many falsely detected labels (the precision G issmall), the label area 13 g with a positive label in the training labelimage 12 is smaller than the label area 13 f with a positive label inthe determination label image 14, and the boundary of the label area 13g is inside the label area 13 f (the outside of the label area 13 g isfalsely detected).

Therefore, when it is determined by the comparison of the non-detectionevaluation value 22 and the false-detection evaluation value 23 that thenumber of falsely detected labels is larger than the number ofundetected labels in the determination label image 14, in step S4 ofcorrecting the label areas 13, the label areas 13 of the training labelimage 12 are expanded.

That is, when 1≈detection rate K>precision G, it can be determined thatthere are few undetected labels and many falsely detected labels.Therefore, the ranges of the label areas 13 of the training label image12 are expanded such that it can be brought closer to the label areas 13of the determination label image 14. In the virtual example of FIG. 13 ,K=1 and G=0.56, and thus the condition (1≈detection rate K>precision G)is satisfied. The criteria for determining the precision G≈1 and thedetection rate K≈1 may be 0.8 or more, or 0.9 or more, for example.

In the first embodiment, a step of excluding the training data 10including the training label image 12 determined to have at least one ofmore than a predetermined threshold number of undetected labels or morethan a predetermined threshold number of falsely detected labels fromthe training data set 1 is further included. When at least one of thenumber of undetected labels or the number of falsely detected labels isgreater than the predetermined threshold, at least one of the detectionrate K or the precision G is below a predetermined threshold Th (K<Thand/or G<Th).

Thus, when one of the non-detection evaluation value 22 and thefalse-detection evaluation value 23 is out of the acceptable range(exceeds the predetermined threshold), the training label image 12 hasextremely large variations in the boundaries of the label areas 13 withrespect to the actual sizes, is considered to be inappropriate, andhinders training. Thus, it is preferable to exclude it from the trainingdata set 1 rather than trying to correct it. The predetermined thresholdcan be set according to the number of classes or 3 the accuracy (lossfunction) of the pre-trained model 2 a. For example, in the case oftwo-class classification, the threshold value can be set to 0.5.

In this manner, the label areas 13 of the training label image 12 arecorrected in step S4. Corrections can be made on the training labelimages 12 of all the training data 10 included in the training data set1. That is, in step S5, it is determined whether or not the traininglabel images 12 of all the training data 10 have been corrected. Whenthere is uncorrected training data (training label image 12), theprocess returns to step S1, and the operations in step S2 to step S4 areperformed on the next selected training data 10. When the correctionprocess for the training label images 12 of all the training data 10 iscompleted, the correction process for the trained model 2 is completed.In addition, a correction can be made only on a specific training labelimage 12. For example, thresholds for determining whether or not acorrection is made are set for the loss function or the non-detectionevaluation value 22 and the false-detection evaluation value 23 suchthat a correction can be made only on a specific training label image 12having the loss function, or the non-detection evaluation value 22 andthe false-detection evaluation value 23 that exceed the thresholds.

In the first embodiment, the label areas 13 of the training label image12 can be corrected once or a plurality of times.

That is, in one aspect of the first embodiment, the step of creating thedetermination label image 14 by the trained model 2 using the trainingdata 10 including the corrected training label image 12 (correctedtraining label image 12 a) is further included, and in the step ofcorrecting the label areas, the label areas 13 included in the correctedtraining label image 12 a are corrected again based on the comparisonresults between the created determination label image 14 and thecorrected training label image 12 a. The details are described below.

(Trained Model Creation Method)

A method for creating the trained model 2 to perform image segmentationusing the training label image (corrected training label image 12 a)corrected by the training label image correction method is nowdescribed.

The trained model creation method includes the following steps:

-   -   (1) a step of acquiring the pre-trained model 2 a by machine        learning using the training data 10 including the input image 11        and the training label image 12,    -   (2) a step of performing the segmentation process on the input        image 11 of the training data 10 by the acquired pre-trained        model 2 a to create the determination label image 14 divided        into the plurality of label areas 13,    -   (3) a step of comparing the labels of the corresponding portions        in the created determination label image 14 and the training        label image 12 with each other,    -   (4) a step of correcting the label areas 13 included in the        training label image 12 based on the label comparison results,        and    -   (5) a step of creating the trained model 2 by performing machine        learning using the training data 10 including the corrected        training label image 12 a.

FIG. 14 shows a process flow for executing the trained model creationmethod. In step S11, the pre-trained model 2 a is acquired. In step S12,the training label image 12 included in the training data set 1 iscorrected. That is, the training label image 12 of each training data 10is corrected by performing the process operations in step S1 to step S5shown in FIG. 10 . Thus, in step S13, the corrected training data set 1including the corrected training label image 12 a is acquired.

In step S14, machine learning is performed using the training data set 1including the corrected training label image 12 a. Consequently, thepre-trained model 2 a is additionally trained using the training data 10including the corrected training label image 12 a in which thevariations in the boundaries of the label areas 13 have beensignificantly reduced or prevented such that the trained model 2 isgenerated. In step S16, the created trained model 2 is stored in astorage of a computer. An example in which machine learning (additionaltraining) is performed on the pre-trained model 2 a using the trainingdata set 1 including the corrected training label image 12 a is shown,but using the training data set 1 including the corrected training labelimage 12 a, machine learning may be performed on another trained modeldifferent from the pre-trained model 2 a or an untrained training model.

The trained model 2 can be created repeatedly a predetermined number oftimes. At this time, the correction process for the training label image12 is also repeatedly performed. That is, in step S16, it is determinedwhether or not the training has been repeated the predetermined numberof times. When the predetermined number of times has not been reached,the process returns to step S11, and the trained model 2 is createdagain.

At this time, in step S11, the trained model 2 stored in immediatelypreceding step S15 is acquired as the pre-trained model 2 a this time.Then, in step S12, the corrected training label image 12 a is correctedagain using the acquired pre-trained model 2 a and the training data set1 including the corrected training label image 12 a.

Then, in step S14, machine learning is performed on the pre-trainedmodel 2 a using the training data set 1 including the re-correctedcorrected training label image 12 a. The operations in step S11 to stepS15 are repeated in this manner such that the trained model 2 is createdby machine learning using the training data set 1 including the updatedand modified training label image 12 a while the training label image 12included in training data set 1 is repeatedly corrected and updated. Instep S16, when the training is repeated the predetermined number oftimes, the process is terminated.

The above training label image correction method and trained modelcreation method are executed by a computer (such as a personal computer,a workstation, or a supercomputer) in which specified software (program)is installed in a storage, or a computer system including a plurality ofcomputers. The computer includes a processor such as a CPU, GPU, or aspecially designed FPGA, and a storage such as a ROM, a RAM, or avolatile or non-volatile storage device (such as an HDD or an SDD).

As shown in FIG. 15 , a processor 50 a of the computer functions as atraining data processor 50 to execute the training label imagecorrection method and the trained model creation method by executingprograms stored in a storage 54.

<Training Data Processor>

The training data processor 50 that executes the training label imagecorrection method according to the first embodiment includes functionalblocks to perform the operations in the steps (step S1 to step S5 inFIG. 10 ) of the training label image correction method, and can executethe training label image correction method by the process using variousdata (the pre-trained model 2 a and the training data set 1) stored inthe storage 54. The training data processor 50 that executes the trainedmodel creation method according to the first embodiment includesfunctional blocks to perform the operations in the steps (step S11 tostep S16 in FIG. 14 ) of the trained model creation method, and canexecute the trained model creation method by the process using variousdata stored in the storage 54.

Specifically, the training data processor 50 includes a determinationimage creating unit 51, a comparator 52, and a label corrector 53 as thefunctional blocks. The training data processor 50 that executes thetrained model creation method according to the first embodiment furtherincludes a training unit 55 as the functional block. When onlycorrecting the training label image 12, the training data processor 50does not need to include the training unit 55. Various data such as thetraining data set 1 and the trained model 2 (pre-trained model 2 a) andprograms are stored in the storage 54.

The training data processor 50 selects the training data 10 in step S1of FIG. 10 . The determination image creating unit 51 is configured tocreate the determination label image 14 by performing the segmentationprocess on the input image 11 of the training data 10 including theinput image 11 and the training label image 12 with the trained model 2stored in the storage 54 as the pre-trained model 2 a. The determinationimage creating unit 51 performs the process operation in step S11 ofFIG. 14 and the process operations in step S1 and step S2 of FIG.

The comparator 52 is configured to compare the labels of thecorresponding portions in the determination label image 14 created bythe determination image creating unit 51 and the training label image 12with each other. That is, the comparator 52 performs the processoperation in step S3 of FIG. 10 .

The label corrector 53 is configured to correct the label areas 13included in the training label image 12 based on the label comparisonresults by the comparator 52. That is, the label corrector 53 performsthe process operation in step S4 of FIG. 10 .

The training data processor 50 repeats the selection of the trainingdata 10 and the correction of the training label image 12 until all thetraining data 10 is selected in step S5 of FIG. 10 .

The training unit 55 is configured to create (update) the trained model2 by performing machine learning using the training data 10 includingthe corrected training label image 12 a. That is, the training unit 55performs the process operations in step S13 to step S15 of FIG. 14 . Thetraining data processor 50 repeats the correction of the training labelimage 12 and the re-training of the trained model 2 the predeterminednumber of times in step S16 of FIG. 14 .

(Image Analysis Device)

An example of an image analysis device that performs image analysis(image segmentation) using the trained model 2 created by the abovetrained model creation method is now described.

(Example of Configuration of Image Analysis Device: Cell AnalysisDevice)

As a first example of the image analysis device, an image analysisdevice 100 shown in FIG. 16 is a cell analysis device that analyzes acell image obtained by imaging a cell.

The image analysis device 100 is configured to perform cell segmentationon an in-line holographic microscopic (IHM) phase image, which is theanalysis image in which a cell has been imaged. The image analysisdevice 100 creates the label image 16 divided for each area of the cellto be detected from the cell image by the segmentation process.

The image analysis device 100 includes an imager 110, a controller 120,an operation unit 130, which is a user interface, a display 140, and thetraining data processor 50.

The imager 110 is an in-line holographic microscope, and includes alight source 111 including a laser diode, for example, and an imagesensor 112. At the time of imaging, a culture plate 113 including a cellcolony (or a single cell) 114 is arranged between the light source 111and the image sensor 112.

The controller 120 is configured to control the operation of the imager110 and process data acquired by the imager 110. The controller 120 is acomputer including a processor and a storage 126, and the processorincludes an imaging controller 121, a cell image creating unit 122, andan image analyzer 123 as functional blocks.

The image analyzer 123 includes an image input 124 that receives aninput of the analysis image 15 and an analysis processor 125 thatperforms a segmentation process for the analysis image 15 using atrained model 2 by machine learning to create a label image divided intoa plurality of label areas 13 as lower functional blocks.

The training data processor 50 has the same configuration as that shownin FIG. 15 , and thus description thereof is omitted. The trained model2 created in the training data processor 50 is stored in the storage 126of the controller 120 and used for the cell segmentation process by theimage analyzer 123.

When the culture plate 113 including the cell colony 114 is set at apredetermined position of the imager 110 by an operator and thecontroller 120 receives a predetermined operation via the operation unit130, the controller 120 controls the imager 110 with the imagingcontroller 121 to acquire hologram data.

Specifically, the imager 110 radiates coherent light from the lightsource 111, and acquires, with the image sensor 112, an image ofinterference fringes of light transmitted through the culture plate 113and the cell colony 114 and light transmitted through an area in thevicinity of the cell colony 114 on the culture plate 113. The imagesensor 112 acquires hologram data (two-dimensional light intensitydistribution data of a hologram formed on a detection surface).

The cell image creating unit 122 calculates phase information byperforming an arithmetic process for phase recovery on the hologram dataacquired by the imager 110. Furthermore, the cell image creating unit122 creates the IHM phase image (analysis image 15) based on thecalculated phase information. Known techniques can be used for thecalculation of the phase information and a method for creating the IHMphase image, and thus detailed description thereof is omitted.

<Cell Image Analyzer>

The cell image shown in FIG. 4 is an example of the IHM phase imagetargeting an undifferentiated deviant cell colony in an iPS cell. In theimage analysis device 100, the segmentation process is performed on theIHM phase image by the image analyzer 123 such that an area divisionprocess of dividing the IHM phase image (analysis image into the labelarea 13 c of the undifferentiated cell, the label area 13 d of theundifferentiated deviant cell, and the label area 13 e of the backgroundis automatically performed.

As shown in FIG. 17 , in step S21, the image input 124 acquires the IHMphase image, which is the analysis image 15. In step S22, the analysisprocessor 125 performs the segmentation process on the analysis image 15by the trained model 2 stored in the storage 126. Thus, the label image16 in which the analysis image 15 input this time has been divided intothe plurality of label areas 13 is created. In step S23, the imageanalyzer 123 stores the created label image 16 in the storage 126, andoutputs the created label image 16 to the display 140 and an externalserver, for example.

In the image analyzer 123, in addition to the segmentation process,various analysis processes based on the results of the segmentationprocess can be performed. For example, the image analyzer 123 estimatesat least one of the area of the cell area, the number of cells, or thedensity of the cells based on the label image 16 obtained by thesegmentation process.

(Example of Configuration of Image Analysis Device: Bone Image AnalysisDevice)

As a second example of the image analysis device, an image analysisdevice 200 shown in FIG. 18 is a bone image analysis device thatanalyzes an X-ray image of an area including the bone of a subject.

The image analysis device 200 is configured to perform bone segmentationon an X-ray image, which is the analysis image 15 in which the bone hasbeen imaged. The image analysis device 200 creates the label image 16 ofthe bone obtained by dividing a bone area from the X-ray image by thesegmentation process.

The image analysis device 200 includes an imager 210, a controller 220,an operation unit 230, a display 240, and the training data processor50.

The imager 210 includes a table 211, an X-ray irradiator 212, and anX-ray detector 213. The table 211 is configured to support a subject O(person). The X-ray irradiator 212 is configured to irradiate thesubject O with X-rays. The X-ray detector 213 includes a flat paneldetector (FPD), for example, and is configured to detect X-rays radiatedfrom the X-ray irradiator 212 and transmitted through the subject O.With the imager 210, for example, as shown in FIG. 2 , the X-ray imageof the area including the bone of the subject is acquired as theanalysis image 15.

The controller 220 is a computer including a processor and a storage226, and the processor includes an imaging controller 221, an X-rayimage creating unit 222, and an image analyzer 223 as functional blocks.

The image analyzer 223 includes an image input 224 and an analysisprocessor 225 as lower functional blocks, similarly to the example shownin FIG. 16 .

The training data processor 50 has the same configuration as that shownin FIG. 15 , and thus description thereof is omitted. The trained model2 created in the training data processor 50 is stored in the storage 226of the controller 220 and used for the bone segmentation process by theimage analyzer 223.

The training label image 12 for performing the bone segmentation processcan be created using the CT image (computed tomography image) data ofthe subject in addition to the X-ray image. Specifically, a virtualprojection is performed on the CT image data of the subject bysimulating the geometrical conditions of the X-ray irradiator 212 andthe X-ray detector 213 to generate a plurality of DRR images (digitalreconstruction simulation images) showing the area including the bone,and an area in which a CT value is above a certain level is labeled asthe bone area such that the training label image 12 can be created. Thetraining data processor 50 can acquire a CT image from a CT imagingdevice (not shown) and generate the training label image 12 based on theacquired CT image.

As shown in FIG. 17 , in step S21, the image input 224 acquires theX-ray image, which is the analysis image 15. In step S22, the analysisprocessor 225 performs the segmentation process on the analysis image 15by the trained model 2 stored in the storage 226. Thus, the label image16 in which the analysis image 15 input this time is divided into theplurality of label areas 13 is created. In step S23, the image analyzer223 stores the created label image 16 in the storage 226 and outputs thecreated label image 16 to the display 240 and an external server, forexample.

In the image analyzer 223, in addition to the segmentation process,various analysis processes based on the results of the segmentationprocess can be performed. For example, the image analyzer 223 estimatesthe density of the detected bone based on the label image 16 (see FIG. 3) obtained by the segmentation process.

In FIGS. 15, 16, and 18 , instead of constructing the training dataprocessor 50 and each of the controllers 120 and 220 as functionalblocks achieved by the processor executing software, the training dataprocessor 50 and each of the controllers 120 and 220 may be constructedby dedicated hardware for performing each process.

Advantages of First Embodiment

In the first embodiment, the following advantages are obtained.

In the first embodiment, as described above, the determination labelimage 14 is created, the labels of the corresponding portions in thecreated determination label image 14 and the training label image 12 arecompared with each other, and the label areas 13 included in thetraining label image 12 are corrected based on the label comparisonresults. Thus, the label areas 13 of the training label image 12 arecorrected based on the comparison results between the determinationlabel image 14 and the training label image 12 with reference to thedetermination label image 14, and thus the variations in the boundariescan be reduced based on consistent criteria by automatic correctionusing a computer instead of a label correction operator. Accordingly,correction of the training label image 12 used for machine learning ofthe image segmentation process can be simplified while ensuring theaccuracy.

In the first embodiment, as described above, in the step of correctingthe label areas 13, the ranges of the label areas 13 in the traininglabel image 12 are corrected so as to be closer to the label areas 13 ofthe determination label image 14 based on the label comparison results.Accordingly, the ranges of the label areas 13 in the training labelimage 12 are simply matched with the label areas 13 of the determinationlabel image 14 or are simply set to intermediate ranges between thelabel areas 13 of the training label image 12 and the label areas 13 ofthe determination label image 14, for example, such that a correctioncan be easily and effectively made to reduce the variations in the labelareas 13 of the individual training label image 12.

In the first embodiment, as described above, in the step of correctingthe label areas 13, correction of the label areas 13 is performed by atleast one of expansion or contraction of the label areas 13 in thetraining label image 12. Accordingly, the label areas 13 of the traininglabel image 12 can be corrected by a simple process of expanding orcontracting the ranges of the label areas 13 in the training label image12 based on the label comparison results.

In the first embodiment, as described above, in the step of correctingthe label areas 13, the range of the label area 13 to which the label ofinterest has been assigned is corrected based on the matching portionand the non-matching portion with the training label image 12 in thedetermination label image 14. Accordingly, it is possible to understand,from the matching portions and the non-matching portions between thedetermination label image 14 and the training label image 12, how thelabel area 13 of the training label image 12 deviates from the referencedetermination label image 14. Therefore, the label area 13 can be easilycorrected based on the matching portion and the non-matching portionsuch that the variation in the boundary is reduced.

In the first embodiment, as described above, in the step of comparingthe labels, the non-detection evaluation value 22 for evaluatingundetected labels in the determination label image 14 and thefalse-detection evaluation value 23 for evaluating falsely detectedlabels in the determination label image 14 are acquired based on thematching portion and the non-matching portion, and in the step ofcorrecting the label areas 13, the label areas 13 of the training labelimage 12 are corrected based on the comparison between the non-detectionevaluation value 22 and the false-detection evaluation value 23. Thus,the non-detection evaluation value 22 and the false-detection evaluationvalue 23 are compared such that it is possible to understand how thelabel areas 13 of the training label image 12 should be corrected(whether it should be increased or decreased), and thus the label areas13 of the individual training label image 12 can be more appropriatelycorrected.

In the first embodiment, as described above, when it is determined bycomparing the non-detection evaluation value 22 with the false-detectionevaluation value 23 that there are more falsely detected labels than theundetected labels in the determination label image 14, in the step ofcorrecting the label areas 13, the label areas 13 of the training labelimage 12 are expanded. Accordingly, when there are many falsely detectedareas and it is estimated that the label areas 13 of the training labelimage 12 are smaller than the label areas 13 of the referencedetermination label image 14, the label areas 13 of the training labelimage 12 are expanded such that a correction can be easily made toreduce the variations in the boundaries of the label areas 13.

In the first embodiment, as described above, when it is determined bycomparing the non-detection evaluation value 22 with the false-detectionevaluation value 23 that there are more undetected labels than thefalsely detected labels in the determination label image 14, in the stepof correcting the label areas 13, the label areas 13 of the traininglabel image 12 are contracted. Accordingly, when there are manyundetected areas and it is estimated that the label areas 13 of thetraining label image 12 are larger than the label areas 13 of thereference determination label image 14, the label areas 13 of thetraining label image 12 are contracted such that a correction can beeasily made to reduce the variations in the boundaries of the labelareas 13.

In the first embodiment, as described above, the step of excluding thetraining data 10 including the training label image 12 determined tohave at least one of more than a predetermined threshold number ofundetected labels or more than a predetermined threshold number offalsely detected labels from the training data set is further included.Accordingly, when at least one of the number of undetected labels or thenumber of falsely detected labels is excessively large in the comparisonwith the reference determination label image 14, the training labelimage 12 is considered to be inappropriate as the training data 10.Therefore, the training label image 12 in which at least one of thenumber of undetected labels or the number of falsely detected labels islarge is excluded from the training data set such that the training data10 that is a factor that lowers the accuracy of machine learning can beexcluded. Thus, the quality of the training data set can be improved,and highly accurate machine learning can be performed.

In the first embodiment, as described above, in the step of comparingthe labels, the labels of the corresponding portions of thedetermination label image 14 and the training label image 12 arecompared with each other for each image pixel. Accordingly, thedetermination label image 14 and the training label image 12 can becompared with each other on a pixel-by-pixel basis, and thus thevariations in the label areas 13 of the training label image 12 can beevaluated more accurately.

In the first embodiment, as described above, the step of creating thedetermination label image 14 by the trained model 2 using the trainingdata 10 including the corrected training label image (corrected traininglabel image 12 a) is further included, and in the step of correcting thelabel areas 13, the label areas 13 included in the corrected traininglabel image 12 a are corrected again based on the comparison resultsbetween the created determination label image 14 and the correctedtraining label image 12 a. Accordingly, machine learning is performedusing the corrected training label image 12 a, and then thedetermination label image 14 is created again such that the label areas13 of the training label image 12 can be repeatedly corrected. The labelareas 13 are repeatedly corrected such that the variations in the labelareas 13 of the training label image 12 can be further reduced.

In the trained model creation method according to the first embodiment,as described above, the label areas 13 included in the training labelimage 12 are corrected based on the label comparison results between thedetermination label image 14 and the training label image 12.Accordingly, the correction of the training label image 12 used formachine learning of the image segmentation process can be simplifiedwhile ensuring the accuracy. Furthermore, the trained model 2 is createdby performing machine learning on the pre-trained model 2 a using thetraining data 10 including the corrected training label image 12 a, andthus the trained model 2 with significantly reduced determinationvariations in the boundary portions of the label areas 13, which iscapable of a high-quality segmentation process, can be obtained.

In the image analysis devices 100 and 200 according to the firstembodiment, as described above, the determination image creating unit 51configured to create the determination label image 14, the comparator 52configured to compare the labels of the corresponding portions in thedetermination label image 14 created by the determination image creatingunit 51 and the training label image 12 with each other, and the labelcorrector 53 configured to correct the label areas 13 included in thetraining label image 12 based on the label comparison results by thecomparator 52 are provided. Accordingly, the correction of the traininglabel image 12 used for machine learning of the image segmentationprocess can be simplified while ensuring the accuracy.

Second Embodiment

A training label image correction method according to a secondembodiment is now described with reference to FIGS. 19 and 20 . In thesecond embodiment, an example is described in which the amount ofcorrection is adjusted according to the label priorities when acorrection is made by expanding or contracting label areas in additionto the training label image correction method according to the firstembodiment.

As shown in FIG. 19 , the training label image correction methodaccording to the second embodiment further includes a step of settingpriorities 60 between labels of a training label image 12. Furthermore,in a step of correcting label areas 13, the amounts of correction of thelabel areas 13 are adjusted according to the priorities 60.

For example, FIG. 19 shows an example in which three labels of anundifferentiated cell, an undifferentiated deviant cell, and abackground are set on a cell image of a pluripotent stem cell, and thecell image is divided by a segmentation process into label areas 13 towhich the three labels have been assigned.

This cell image is used to prevent an undifferentiated cell thatmaintains pluripotency from differentiating by finding anundifferentiated deviant cell in a cell colony and removing it, and toculture only the undifferentiated cell when a pluripotent stem cell iscultured. That is, it is desired to reliably find an undifferentiateddeviant cell, and thus among the labels of an undifferentiated cell, anundifferentiated deviant cell, and a background, the priorities 60 areset higher in the order of a background, an undifferentiated cell, andan undifferentiated deviant cell.

In the second embodiment, in the step of correcting the label areas 13,when the label areas 13 of the training label image 12 are corrected,expansion of the label areas 13 having a lower priority to the labelareas 13 having a higher priority is prohibited. That is, the amounts ofcorrection of the label areas 13 are adjusted according to thepriorities 60 such that the label areas 13 having a higher priority arenot reduced by the expansion of the label areas 13 having a lowerpriority.

In an example of FIG. 20 , when the label areas 13 are corrected byexpansion, expansion of a label area 13 e of the background to a labelarea 13 c of the undifferentiated cell and a label area 13 d of theundifferentiated deviant cell is prohibited. Furthermore, expansion ofthe label area 13 c of the undifferentiated cell to the label area 13 dof the undifferentiated deviant cell is prohibited.

On the other hand, when the label areas 13 are corrected by expansion,expansion of the label area 13 d of the undifferentiated deviant cell tothe label area 13 c of the undifferentiated cell and the label area 13 eof the background is allowed. Furthermore, expansion of the label area13 c of the undifferentiated cell to the label area 13 e of thebackground is allowed. Consequently, in the second embodiment, in thetraining label image 12, a correction in a direction in which the labelareas 13 having a label with a higher priority are reduced isrestricted.

The remaining configurations of the second embodiment are similar tothose of the first embodiment. In another configuration example of thesecond embodiment, expansion of the label areas 13 having a lowerpriority to the label areas 13 having a higher priority may not beprohibited. For example, the amount of expansion from the label area 13c having a lower priority to the label area 13 d having a higherpriority is smaller than the amount of expansion from the label area 13d having a higher priority to the label area 13 c having a lowerpriority. Even in this manner, the correction in the direction in whichthe label areas 13 having a label with a higher priority are reduced canbe restricted.

Advantages of Second Embodiment

In the second embodiment, the following advantages are obtained.

In the second embodiment, similarly to the first embodiment describedabove, correction of the training label image 12 used for machinelearning of the image segmentation process can be simplified whileensuring the accuracy.

In the second embodiment, as described above, the step of setting thepriorities 60 between the labels of the training label image 12 isfurther included, and in the step of correcting the label areas 13, theamounts of correction of the label areas 13 are adjusted according tothe priorities 60. Accordingly, the amount of correction of a label witha higher priority can be intentionally biased such that detectionomissions can be significantly reduced or prevented as much as possible.Consequently, the training label image 12 can be appropriately correctedaccording to the purpose of the segmentation process.

In the second embodiment, as described above, in the step of correctingthe label areas 13, when the label areas 13 of the training label image12 are corrected, expansion of the label areas 13 having a lowerpriority to the label areas 13 having a higher priority are prohibited.Accordingly, even when the label areas 13 having a lower priority (thelabel area 13 c, for example) are corrected by expansion, the labelareas 13 having a higher priority (the label area 13 d, for example) canbe preserved. Therefore, it is possible to obtain the training labelimage 12 (training data 10) capable of being learned, in which detectionomissions of the label areas 13 with a higher priority are significantlyreduced or prevented while variations in the boundaries aresignificantly reduced or prevented according to the purpose of thesegmentation process.

Modified Examples

The embodiments disclosed this time must be considered as illustrativein all points and not restrictive. The scope of the present invention isnot shown by the above description of the embodiments but by the scopeof claims for patent, and all modifications (modified examples) withinthe meaning and scope equivalent to the scope of claims for patent arefurther included.

For example, while as examples of an image, the cell image and the X-rayimage have been shown in the aforementioned embodiment, the presentinvention is not limited to this. The correction of the training labelimage according to the present invention can be applied to any image aslong as the same is a training label image used for machine learning ofthe segmentation process. The correction of the training label imageaccording to the present invention is to correct variations in theboundaries of the label areas, and thus it is particularly suitable inthe field in which high accuracy is required at the boundary portions ofthe label areas. As such an image, there is a so-called medical imageparticularly used in the medical field (the healthcare field or themedical science field). The medical image is used to diagnose a diseaseby a doctor, for example, and thus it is desired to significantly reduceor prevent the blurring or the unnatural shapes of the boundaries asmuch as possible even when the boundary portions of the label areas arefine. Therefore, the present invention, which can reduce variations inthe boundaries of the label areas by correcting the training labelimage, is particularly suitable when used for segmentation of a medicalimage. As the medical image, in addition to a cell image and an X-rayimage of a bone and other portions, an image showing a tumor (such as anendoscopic image) in a case in which a segmentation process is performedon a tumor area to be treated as a detection target may be used, forexample.

While the example in which the entire training label image 12 iscompared with the determination label image 14 pixel for each pixel, anda correction is made based on the comparison results has been shown ineach of the aforementioned first and second embodiments, the presentinvention is not limited to this. In the present invention, a portion ofthe training label image 12 may be compared with the determination labelimage 14, and a correction may be made. For example, in a modifiedexample of the training label image correction method shown in FIG. 21 ,in a step of correcting label areas 13, a training label image 12 isdivided into a plurality of partial images 17, and the label areas 13are corrected for at least one partial image 17 of the divided traininglabel image 12. In FIG. 21 , the training label image 12 is divided intoa matrix so as to have 6×6 rectangular partial images 17 a. Then, in themodified example, a determination label image 14 is also divided intopartial images 17 b, the training label image 12 and the determinationlabel image 14 are compared with each other for each partial image 17(17 a, 17 b), and the partial images 17 of the training label image 12are corrected based on the comparison results. The shape of each partialimage 17 and the number of divisions are not particularly limited andare arbitrary. The correction may be made only for specific one or morepartial images 17, or may be made for all the partial images 17. Withthe configuration as in this modified example, only a specific portionof the training label image 12 can be corrected, or each small portioncan be corrected individually.

While the example in which the image analysis devices 100 and 200perform the correction process on the training label image 12 has beenshown in the aforementioned first embodiment, the present invention isnot limited to this. The correction process for the training label image12 may be performed by another device other than the image analysisdevices. Similarly, while the example in which the image analysisdevices 100 and 200 create the trained model 2 by machine learning hasbeen shown in each of the aforementioned first and second embodiments,the trained model 2 may be created by another device other than theimage analysis devices.

For example, in a modified example shown in FIG. 22 , a training dataprocessor 50 is provided as a server device separate from an imageanalysis device 300, and is connected to the image analysis device 300via a network. The training data processor 50 acquires, from the imageanalysis device 300, a newly captured analysis image 15 as an inputimage 11 of training data 10 for machine learning. A training labelimage 12 is created from the input image 11 by the training dataprocessor 50 or using another image processing device, and the trainingdata 10 including the input image 11 and the training label image 12 isadded to a training data set 1. The training data processor 50 creates adetermination label image 14 from the input image 11 by a pre-trainedmodel 2 a, compares it with the training label image 12, and correctsthe training label image 12 based on the comparison results. Thetraining data processor 50 re-trains the pre-trained model 2 a using thecorrected training label image 12 and creates (updates) a trained model2. The training data processor 50 transmits the newly created trainedmodel 2 to the image analysis device 300 via the network. Thus, theimage analysis device 300 can perform a segmentation process using thenewly created trained model 2.

Furthermore, for example, in a modified example shown in FIG. 23 , atraining label image 12 is corrected in an image analysis device 300. Atrained model 2 is created on the training data processor 50 side. Onthe image analysis device 300 side, the training label image 12 iscreated using a newly captured analysis image 15 as an input image 11 oftraining data 10. The training data 10 including the input image 11 andthe training label image 12 is added to a training data set 1. The imageanalysis device 300 creates a determination label image 14 from theinput image 11 by a pre-trained model 2 a, compares it with the traininglabel image 12, and corrects the training label image 12 based on thecomparison results. The image analysis device 300 transmits the trainingdata 10 including the input image 11 and a corrected training labelimage 12 a to the training data processor 50 via a network. The trainingdata processor 50 adds the transmitted training data 10 to the trainingdata set 1, re-trains the pre-trained model 2 a, and creates (updates)the trained model 2. The training data processor 50 transmits the newlycreated trained model 2 to the image analysis device 300 via thenetwork. Thus, the image analysis device 300 can perform a segmentationprocess using the newly created trained model 2.

Thus, the training label image correction method and the trained modelcreation method according to the present invention may be executed inthe form of a so-called cloud service or the like by the cooperation ofa plurality of computers connected to the network.

While the example in which the label areas 13 of the training labelimage 12 are corrected by the morphology process has been shown in theaforementioned first embodiment, the present invention is not limited tothis. In the present invention, the label areas may be corrected by amethod other than the morphology process. For example, when the numberof undetected or falsely detected labels is greater than a predeterminedthreshold, the label areas 13 of the training label image 12 may bereplaced with the label areas 13 of the determination label image 14.Alternatively, for example, the boundaries of the label areas 13 of thecorrected training label image 12 a may be corrected so as to be locatedbetween the boundaries of the label areas 13 of the training label image12 and the boundaries of the label areas 13 of the determination labelimage 14. In these cases, it is not always necessary to use thenon-detection evaluation value 22 and the false-detection evaluationvalue 23 unlike the first embodiment.

While the example in which the detection rate K shown in the aboveformula (1) is used as the non-detection evaluation value 22, and theprecision G shown in the above formula (2) is used as thefalse-detection evaluation value 23 has been shown in the aforementionedfirst embodiment, the present invention is not limited to this. Forexample, instead of either the detection rate K or the precision G,Intersection over Union (IoU) shown in the following formula (3) or an Fvalue shown in the following formula (4) may be used.IoU=TP/(—+FN+FP)  (3)F value=2K×G/(K+G)  (4)

Note that K in the above formula (4) is a detection rate shown in theabove formula (1), and G is a precision shown in the above formula (2).

These IoU and F value are complex indexes for evaluating both undetectedand falsely detected labels, and correspond to both the non-detectionevaluation value 22 and the false-detection evaluation value 23. The IoUor F value is used in combination with the detection rate K or theprecision G such that undetected labels and falsely detected labels canbe evaluated separately. For example, when IoU and the detection rate Kare used, it is determined that there are few falsely detected labelsand many undetected labels when K≈IoU<<1, and it is determined thatthere are few undetected labels and many falsely detected labels when1≈K>IoU.

In the first embodiment, the non-detection evaluation value 22 and thefalse-detection evaluation value 23 have been described on the premiseof the two-class confusion matrix 21 shown in FIG. 11 , but the same arealso applicable to a multi-class segmentation process of three or moreclasses. Although the description is omitted, even in the case of threeor more classes, the non-detection evaluation value and thefalse-detection evaluation value such as the detection rate and theprecision may be obtained from a confusion matrix according to thenumber of classes.

DESCRIPTION OF REFERENCE NUMERALS

-   -   1: training data set    -   2: trained model    -   2 a: pre-trained model    -   10: training data    -   11: input image    -   12: training label image    -   12 a: corrected training label image    -   13, 13 a, 13 b, 13 c, 13 d, 13 e, 13 f, 13 g: label area    -   14: determination label image    -   15: analysis image    -   16: label image    -   17, 17 a, 17 b: partial image    -   22: non-detection evaluation value    -   23: false-detection evaluation value    -   51: determination image creating unit    -   52: comparator    -   53: label corrector    -   54: storage    -   60: priority    -   100, 200: image analysis device    -   124, 224: image input    -   125, 225: analysis processor    -   126, 226: storage

The invention claimed is:
 1. A method for generating training data setsfor machine learning comprising: preparing a pre-trained model forperforming a segmentation process the pre-trained model trained based ontraining data sets, each of the training data sets including an inputimage and a training label image, the training label image including aplurality of divided first label areas, performing the segmentationprocess on the input image for one of the training data sets to output adetermination label image including a plurality of divided second labelareas; comparing the first label areas of the training label image andthe second label areas of the determination label image; and correctingthe first label areas based on the step of comparing the first andsecond label areas.
 2. The method for generating the training data setsfor the machine learning according to claim 1, wherein the correctingthe first label areas includes correcting ranges of the first labelareas in the training label image so as to be closer to the second labelareas of the determination label image based on a label comparisonresult.
 3. The method for generating the training data sets for themachine learning according to claim 2, wherein the correcting the firstlabel areas includes correcting the first label areas by at least one ofexpansion or contraction of the first label areas in the training labelimage.
 4. The method for generating the training data sets for themachine learning according to claim 1, wherein: the comparing the firstlabel areas of the training label image and the second label areas ofthe determination label image includes acquiring a matching portion anda non-matching portion with the training label image in thedetermination label image for a label of interest by comparing thedetermination label image with the training label image; and thecorrecting the first label areas includes correcting a range of a firstlabel area with the label of interest based on the matching portion andthe non-matching portion.
 5. The method for generating the training datasets for the machine learning according to claim 4, wherein: thecomparing the first label areas of the training label image and thesecond label areas of the determination label image includes acquiring anon-detection evaluation value for evaluating undetected labels in thedetermination label image and a false-detection evaluation value forevaluating falsely detected labels in the determination label imagebased on the matching portion and the non-matching portion; and thecorrecting the first label areas includes correcting the first labelareas of the training label image based on a comparison between thenon-detection evaluation value and the false-detection evaluation value.6. The method for generating the training data sets for the machinelearning according to claim 5, wherein the correcting the first labelareas includes expanding the first label areas of the training labelimage when it is determined by the comparison between the non-detectionevaluation value and the false-detection evaluation value that there aremore falsely detected labels than undetected labels in the determinationlabel image.
 7. The method for generating the training data sets for themachine learning according to claim 5, wherein the correcting the firstlabel areas includes contracting the first label areas of the traininglabel image when it is determined by the comparison between thenon-detection evaluation value and the false-detection evaluation valuethat there are more undetected labels than falsely detected labels inthe determination label image.
 8. The method for generating the trainingdata sets for the machine learning according to claim 5, furthercomprising: excluding the training data including the training labelimage determined to have at least one of more than a predeterminedthreshold number of undetected labels or more than a predeterminedthreshold number of falsely detected labels from training data set. 9.The method for generating the training data sets for the machinelearning according to claim 1, further comprising: setting prioritiesbetween first labels of the training label image; wherein the correctingthe first label areas includes adjusting amounts of correction of thefirst label areas according to the priorities.
 10. The method forgenerating the training data sets for the machine learning according toclaim 9, wherein the correcting the first label areas includescorrecting the first label areas of the training label image whileexpansion of the first label areas having a lower priority to the firstlabel areas having a higher priority is prohibited.
 11. The method forgenerating the training data sets for the machine learning according toclaim 1, wherein the correcting the first label areas includes dividingthe training label image into a plurality of partial images, andcorrecting the first label areas for at least one partial image of thedivided training label image.
 12. The method for generating the trainingdata sets for the machine learning according to claim 1, wherein thecomparing the first label areas of the training label image and thesecond label areas of the determination label image includes comparingthe first and second labels of the corresponding portions of thedetermination label image and the training label image with each otherfor every image pixel or every plurality of adjacent image pixels. 13.The method for generating the training data sets for the machinelearning according to claim 1, further comprising: creating thedetermination label image by the trained model using the training dataincluding the corrected training label image; wherein the correcting thefirst label areas includes correcting the first label areas included inthe corrected training label image again based on comparison resultsbetween the created determination label image and the corrected traininglabel image.
 14. A trained model creation method by machine learning forperforming a segmentation process on an image, the trained modelcreation method comprising: preparing a pre-trained model for performingsegmentation process, the pre-trained model trained based on trainingdata sets, each of the training data sets including an input image and atraining label image, the training label image including a plurality ofdivided first label areas; performing the segmentation process on theinput image for one of the training data sets by the preparedpre-trained model to output a determination label image including aplurality of divided second label areas; comparing first and secondlabels of corresponding portions in the created determination labelimage and the training label image with each other; correcting the firstlabel areas, based on the step of comparing the first and second labelareas; and creating a trained model by performing the machine learningusing the training data including the corrected training label image.15. An image analysis device comprising: an image input configured toreceive an input of an analysis image; an analysis processor configuredto perform a segmentation process on the analysis image using a trainedmodel by machine learning to create a label image divided into aplurality of label areas; a storage configured to store the trainedmodel; a determination image creating unit configured to perform thesegmentation process on an input image for one of training data setsincluding the input image and a training label image including aplurality of divided first label areas with the trained model stored inthe storage as a pre-trained model to create a determination label imageincluding a plurality of divided second label areas; a comparatorconfigured to compare the first and second labels of correspondingportions in the determination label image created by the determinationimage creating unit and the training label image with each other; and alabel corrector configured to correct the first label areas based ionthe comparing the first and second label areas by the comparator.